Second Law of Nonequilibrium Thermodynamics at Different Levels of Description 
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In the case where the dynamics of a system may be described by two or more stochastic processes at different 
scales, the total entropy production, the sum of Shannon entropy increment in the system and entropy produc- 
tion due to heat transfer, may depend on the levels of description. By analyzing general Markov stochastic 
processes with a parameter representing the extent of separation of time scales, we find that the excess entropy 
production, a key quantity in steady state thermodynamics, is invariant with respect to the change in the level 
of description. We demonstrate our findings in a two-dimensional overdamped Langevin model where the total 
entropy production differs between the levels. These results suggest that the second law-like relations involving 
the excess entropy production are level-independent relations even in the nonequilibrium setup, and therefore 
act as promising alternatives to the ordinary second law. 
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Introduction. — Since Einstein's study on Brownian mo- 
tion [Q]], the applicability of thermodynamics has been ex- 
tended to fluctuating worlds. In the framework of small sys- 
tems thermodynamics, several thermodynamic quantities such 
as heat, work and entropy are naturally defined, and it can be 
verified that they satisfy the mesoscopic version of thermo- 
dynamic relations ||2|]. Recent development of experimental 
techniques enable the measurement of fluctuating quantities 
in various setups including biological molecular motors fl, 
mesoscopic quantum systems I4J and information processings 
which makes more important the interpretation of measur- 
able quantities from the thermodynamic point of view. 

In the mesoscopic level, a system is often modeled in var- 
ious ways where the dynamics is described by different sets 
of variables corresponding to which subsystem is focused on. 
Accordingly, heat, which is essentially the energy transfer to 
the hidden degrees of freedom [6], changes its value depend- 
ing on the levels of description. Sekimoto discussed the rela- 
tion between the heat at different levels, for a mesoscopic sys- 
tem that consists of a slow variable x, a fast variable y, and the 
environment [7]. It was pointed out that the relation between 
the two heat, Q x,y and Q x , corresponding to the (x, y)-level and 
the x-level descriptions, respectively, is analogous to that be- 
tween the effective Hamiltonians in the equilibrium statistical 
mechanics. It follows from the analogy that the total entropy 
production, which is the sum of the Shannon entropy incre- 
ment in the system of interest and the heat divided by tem- 
perature of the environment, has level-independence and ob- 
jectivity in the sense defined in |7|. The physical meaning of 
this independence is that the difference in the heat, Q x ' y - Q x , 
amounts to the heat exchanged reversibly with the environ- 
ment (reversible heat). The above statement follows the intu- 
ition that when the fast variable y is thermalized in the slow 
time scale of the x-dynamics, y could be regarded as a part of 
the environment. 

In contrast to the preceding argument, it has been shown 
that in a certain non-equilibrium setup, the total entropy pro- 
duction may change its value depending on the levels of de- 
scription, even when the separation of time scale justifies the 



elimination of the fast variables from the dynamics @]. Such 
difference in the total entropy production, which we call the 
hidden entropy production [9], is expected to be commonly 
observed in non-equilibrium systems, implying that in a gen- 
eral setup, the physical laws involving the entropy production, 
such as the second law of thermodynamics, is not invariant 
with respect to the change in the levels of description. It is 
necessary for solving this problem to describe the physical 
laws by using quantities that do not depend on the levels. 

In the present Letter, we focus on the excess part of entropy 
production which is a key quantity to construct steady 

state thermodynamics (SST), a possible extension of thermo- 
dynamics to nonequilibrium steady state. We show for general 
Markov processes and several definitions of excess entropy 
productions 012l4l4ll that the sum of the system's entropy in- 
crement and the excess entropy production is kept invariant 
between the different levels of description, even when the hid- 
den entropy production exists. These invariances guarantee 
the objectivity of second laws of nonequilibrium thermody- 
namics. 

Invariance of excess entropy production. — Let us con- 
sider a general Markov process following the master equation, 



dP,(x,y) 
dt 



= f£-P t (x,y). 



(1) 



Here, P, is the probability density function, x and y represent 
slow and fast variables, respectively, £*f is the generator of 
time evolution which depends on time-dependent external pa- 
rameters denoted by A,. By integrating ([1) with respect to y, 
we may formally write the reduced master equation as 



dP,(x) 
dt 



L\ ■ P,(x). 



(2) 



Equation (O gives the closed Markov dynamics of x if the 
reduced generator of time evolution, £, X A - f -C*^ ■ pfv dy, 
depends only on A, and not explicitly on t . Denoting the time 
scale of the x,y dynamics and A, modulation as r A , t v and Tx, 
respectively, this condition is satisfied when the parameter of 
separation of time scale, r\ : = max{r v /r t , Ty/rj}, is sufficiently 
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small, and the probability density for y relaxes to the station- 
ary distribution under given x and A,, 



P t (x,y) = P A '(y\x)P l (x) + 0(7 ] ). 



(3) 



In several systems modeled as Markov processes lfl5l[l6ll . 
it has been considered that the entropy production due to the 
energy transfer to the hidden degrees of freedom is given by 
the transition probability W A '(x'|x) from x to x' between time 
f and t + At as, 



cr(t,A t ) := lim — 

A(^() Af 



In 



w^;(x, +Af |x f ) 
wf;(x,|x, +Af ) 



(4) 



at both levels of description, x = (x,y) and x. Here, (•) rep- 
resents the average over x, and x, +A , at times f and t + At, 
x indicates the time-reversal of x, and the Boltzmann con- 
stant is set to unity. We call this entropy production with a 
prefix "ordinary" to avoid ambiguity. To take into account 
the reversible heat, we define the Shannon entropy of x as 
S[P t (x)] := — f P,(x)lnP,(x)dx, and consider the total en- 
tropy production, 



T,(t, A,) := S [P,(x)] + cr(t, A,) > 0. 



(5) 



Here, indicates the time derivative. Inequality (0 holds for 
an arbitrary P,(x) and shall be interpreted as the mesoscopic 
version of the second law. We later show through an explicit 
example that in certain cases not only the entropy production, 
cr(t, A t ), but also the total entropy production, £(f, A t ), may 
have different values in the original and reduced dynamics, 
even in the limit of 77 — > 0. 

Our main approach is to investigate the dependence of en- 
tropy production-like quantities on the levels of description. 
The key quantity we consider is the excess entropy produc- 
tion, defined as the entropy production from which the steady 
state housekeeping part is subtracted. There are many types of 
excess entropy production corresponding to the different def- 
initions of the housekeeping part, which, in all cases, reduce 
to the ordinary entropy production in equilibrium dynamics, 
and are equal to zero at the steady state. First, we consider 
the excess entropy production introduced by Hatano and Sasa 
EI, 



0~ex(f, ^f) - z 



Af-*0 Af \ Pjl( Xt+At )l 



(6) 



f S s(x) is the steady state density function under A,. For 
any Markovian stochastic dynamics 17] (originally shown in 
overdamped Langevin dynamics |72]), cr ex (t, A,) satisfies, for 
an arbitrary P,(x), the Hatano-Sasa inequality, 



S[P,(x)] + CTex(Ur)>0. 



(7) 



This inequality may be considered as an extension of the sec- 
ond law to a nonequilibrium setup, the equality of which is 
achieved in steady state, whereas that of Eq. © is achieved 



only in equilibrium state. When the condition Eq. (O holds, 
the excess entropy productions, cr^lit, A t ) and <r* x (t, A t ), cor- 
responding to two levels [Eq. (Q]l and Eq. ©], are related as, 

S [P,(x,y)] + o™(t, A,) ^ - /in P '+A,fe + Ar,?r + A,)f s^,y,) \ 
Af\ P,(x t ,y t )P A s i(x r+Al ,y t+At ) I 



4n„ 



Pt+At(.Xt+ A t)Pss ( x l) 
P t (Xt)P A s(x, +At ) 



+ U^-lnP i (y,|* f )) 



dA 



S[P,(x)] + o±(t,A t ). 



(8) 



Here, - indicates that the terms O(At), 0(rj) are ignored, 
and we used {Atfj \nP A (y,\x,)) = 0. Equation ([8]) states 
that, although the excess entropy production itself may de- 
pend on the levels of description, the difference cr^(t, A t ) - 
<r* x (t,A,) amounts to the contribution of the reversible heat 
4 (S [P t (x,y)] - S [P t (x)]), and therefore the Hatano-Sasa in- 
equality (0 is kept invariant between the different levels. 

Next, we investigate the excess entropy production adopted 
by Komatsu et al. [13]. They considered the steady state 
entropy production for a fixed parameter A, cr ss(KNSX )(/l) := 
cr(f — > 00, A), and defined the excess entropy production as 



C ex(KNST)(f, = 0~(t, A t ) — (T SS (KNST) ) 



(9) 



1 1311 . In the quasistatic parameter change from Ao to A-j- start- 
ing with the steady state Pp(x), cr ex(K NST)(f, ^r) satisfies the 
extended Clausius relation 11311 . 

Ssymtf-r] - S sym [P ] = - f CTex(KNST)(f, ^)df + O^A), 

Jo 

(10) 

where, S sym [P,(x)] = JdxP,(x)| (lnP,(x) + lnP,(x)) is the 
symmetrized Shannon entropy, e and A are the dimension- 
less quantities which characterize the "degree of nonequilib- 
rium" and the change At - Aq, respectively II 1 311 - Accord- 
ing to the relation between o" e x(KN£i)(f, A t ) and Hatano-Sasa- 
like excess entropy production ll 811 . the sum of the Shan- 
non entropy increment and the excess entropy production, 
S [P r (x)] + cr ex (KNST)(f, A t ), is kept invariant within an error of 
0(e 2 A) for the case where Eq. OJ holds and P A '(y\x) possesses 
the symmetry P A '(y\x) = P A '(y\x). It follows from this invari- 
ance that the extended Clausius relation ( TTOb is invariant with 
respect to the change in the level of description, as 

S sym[Pr(x, y)] - S sym [Po(x, y)] + J 0"ex(KNST)( f ' A,)dt 
= S sym [Pr(x)] - S sym [Po(*)] + I 0"ex(KNST)( f ' A t)&t 

+ 0(e 2 A) + 0(i]), (11) 

under an additional condition, P A '(y\x) = P A '(y\x), which is the 
equivalent condition for a fluctuation theorem of the hidden 
entropy production to hold |20. 

Overdamped Langevin system. — To clarify the existence 
of hidden entropy production and the invariance of excess 
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entropy production, we investigate a two-dimensional over- 
damped Langevin model described by, 

y,(j) = F^,y)+ V^(|l (12) 

Here, index t represents time-dependence of the external pa- 
rameters included in A,, y, is the drag coefficient, T, is the 
temperature of the environment, % x ,gy are independent white 
Gaussian noises with unit variance. The general external force 
¥,(x,y) may be written by the scalar potential U t (x,y; a) and 
the vector potential parallel to the z-direction V t (x,y;a) = 
V t (x,y; a) e z as F, = -VU t - V x V,. The nonconservative 
force, -V x V,, drives the system out of equilibrium. 

The separation of time scale and the strength of the non- 
conservative force in Eq. ( TTZb are controlled by introducing 
the parameter a as 

U t (x,y; a) — U t (x, ay), V t (x,y;a) — a~ l V t (x, ay). (13) 

As a becomes large, the scalar potential U,(x, y; a) squeezes 
the extent of the probability density along the y-direction, and 
the diffusive time scale, r v oc a" 2 , becomes small compared 
with the other time scales, t x and t,\. By the procedure of sin- 
gular perturbation theory 11911 . an arbitrary probability density 
function given at time t — converges to 

P,(x,y) = P,(x) — — + 0(?]), (14) 

Z,(x) 

at t » T y , and consequently Eq. ( fT2l is reduced to the closed 
Langevin dynamics of x, 

Yt* = {t,^ \nZ,(x) + //(*) + ^T,^ [1 + 0(rj)] . (15) 

Derivation is given at the end of the Letter. Z,(x) is the local 
partition function, 



Z t (x) = aj 



exp 



U,(x,y;a) 
T, 



Ay, 



(16) 



and f^(x) is the averaged jc-component of the force that orig- 
inates from the vector potential, 

(17) 

Note that the Langevin equation ( fT~5T > at the reduced level may 
have an equilibrium state even when Eq. (fT2l at the original 
level has no equilibrium state in the limit 77 — > 0. 

We examine the total entropy production at the original 
level [Eq. (TT2l l and the reduced level [Eq. <fT3T >1 . For a 
Langevin system, the entropy production at each level can be 
rewritten as, 



1 1 

(r x (t, A,) = — (-[-y t x + ^ylr^A o x) 



(18) 
(19) 



where o denotes the product in the Stratonovich sense. By 
substituting Eqs. (fT2l . ( fl3T > into Eqs. ( fT8b . ( fT9l and evaluating 
the average using the local stationary density (TBI , we may 
show 

^(UO-^U^-^/^f^-ln^)) 



1 



dV 



_ , - Mx rni + om. 



(20) 



versible heat, 4(5 [P t (x,y)] - S [P,(x)]), therefore the hidden 



The first term of the right hand side is equivalent to the re 

d_ 

it 

entropy production is given by, 
E(t,A,):=r' y (t,A t )-r(t,A t ) 



[i + om, (2D 



which is a non-negative quantity since the right hand side of 
Eq. (f2Tb is the variance of -^grp^ , 

Let us describe here the simplest example, U(x, y; a) = 
u(x) + k(ay) 2 /2 and V(x,y;a) = (Ik/T)Uay) 2 /a. The dy- 
namics of x is described by, 



yx — —u'(x) + f(f) + 



(22) 



where ( = -(Ik/T)iay may be considered as a fluctuating ex- 
ternal force, which behaves as a colored Gaussian noise with 
correlation time t v = y/ka 2 , (g(i)£(s)) = 7exp(-|f - s\/T y ). 
In this (x, y)-level, the entropy production is obtained as, 

o**(f) = y- l {{-u'{x) +0°*) = «-"'(-*) ° -*) + I), (23) 

in the limit of T y — > 0. Now £ in Eq. d22l has no effect on the 
dynamics of x in this limit, since it has no singularity like that 
in % x . Therefore, y may be eliminated from the description of 
dynamics, and yx = -u'(x) + ^2yT £ v is obtained. Entropy 
production in this jc-level is, 



o-\i) = y-\-u'(x)o x ). 



(24) 



The discrepancy, y~ l I, between Eqs. (l23l and d24l i corre- 
sponds to the hidden entropy production, E(t, A t ). This ex- 
ample demonstrates that a hidden force like £ which may be 
present in any setup affects the value of entropy production, 
indicating that the standard method of stochastic energetics 
fails to evaluate the entropy production in a level-independent 
manner. 

We have seen that the hidden entropy production in our 
model is generally positive, even in the limit of 77 — » 0, where 
the reduction of dynamics to Eq. (15[ is completely justified. 
In contrast, the excess entropy productions are kept invariant 
between the levels, since Eq. (fl4l > meets the condition (f3]l in 
the previous general analysis. The invariance of the extended 
Clausius relation ( TTOb may also be directly verified by an order 



4 



estimation of the excess part derived from the hidden entropy 
production, 



I 



r, 2 \\ ^ 



(25) 



Here, we introduced the typical length scale along the in- 
direction l x , to relate the degree of nonequilibrium e := l fj^ 
and the deviation of the probability density distribution from 
the steady distribution, which is proportional to t x /ta with 
t x = Odlyr/T,) and t a = 0(77 A). 

In this model, another definition of the excess entropy pro- 
duction is also kept invariant between the levels. For an 
arbitrary overdamped Langevin system, Maes and Netocny 
found that for fixed parameters except U, a given distribu- 
tion P, is a steady state of the dynamics with the potential 
U = argmin^ H x,y (t, A,). Based on this fact, they defined the 
steady entropy production as cr ss(MN )(f, A,) :- miny l, x ' y (t, A t ) 
lfl4fl . Then, their modified excess entropy production is de- 
fined as 



0" m ex(f, A t ) = <T(t, At) - 0- ss(M N)(f, A t ), 



(26) 



which satisfies another second law-like inequality for an arbi- 
trary P/(x), 



S[P t (x)]+cr mex (t,At)>0, 



(27) 



in the overdamped Langevin system 01411 . It follows from 
Eq. (fJTJ that the left hand sides of the inequality ( f27T > in the 
original level and the reduced level are equivalent in the limit 
of small 77, 

S [P,(x,y)] + cr^ x (t, A,) = ($ [P t (x)] + < ex (f, A,)) [1 + 0(t,)]. 

(28) 

These invariances of the excess entropy productions 
[Eqs. ([8j, ( fTTT i. < f28T >1 indicate that the hidden entropy produc- 
tion (l2"TT i is essentially included in the housekeeping part, not 
in the excess part. 

Remarks. — In the present Letter, we analyzed the two- 
dimensional overdamped Langevin model to exemplify the 
invariance of the excess entropy production. Similar struc- 
tures, of course, exist in other systems; for example, under- 
damped Langevin systems that can be reduced to overdamped 
Langevin dynamics. As originally pointed out in 081], spatially 
inhomogeneous temperature produces difference in the total 
entropy production between the underdamped description and 
the overdamped description (general expression was given by 
ll20ll ). Since Eq. (O is valid in the overdamped limit, we 
can show that the Hatano-Sasa inequality (0 [ 12] and the ex- 
tended Clausius relation (TTOb 1 13] are kept invariant between 
the two descriptions. 

We further comment on the relation between the ordinary 
entropy production and the excess entropy production. In the 
special case where the reduced dynamics has an equilibrium 
state, the excess entropy productions are equivalent to the or- 
dinary entropy production at the reduced level, cr x (t, A t ) - 



o~ex( f > A >) = °"mex( f ' A t) = o"e X(K NST)( f ' A t)- As we have shown, 
these entropy productions are also equivalent to cr x x y (t, A t ) and 
<Xmg X (f, A,). Therefore, the nonequilibrium second law-like in- 
equalities [(0 or (1271 11 in the original level are equivalent to 
the inequality for the total entropy production at the reduced 
level, 57(/, A t ) > 0, unlike the corresponding inequality at the 
original level T, x ' y (t, A,) > 0. 

Conclusion. — We considered general Markovian 
stochastic processes which can be described by two levels of 
description, and showed that the excess entropy productions 
are essentially invariant with respect to the change in the 
levels, even in the case where the hidden entropy production 
exists. These results were investigated in the two-dimensional 
overdamped Langevin model. 

Our results are encouraging for the experimental investiga- 
tion of SST Measurable variables are often restricted to few 
degrees of freedom, due to the limitation in experimental tech- 
niques. In such cases, as we have verified, the total entropy 
production does not generally describe the physical laws in a 
level-independent manner. It is expected, however, that the 
excess entropy production possesses an objective value, when 
the dynamics is closed (in the Markovian sense) among the 
measurable quantities. Therefore, the objectivity of the phys- 
ical laws is recovered by considering the excess entropy pro- 
duction, supporting the possibility of experimentally explor- 
ing SST based on the mesoscopic level of description. 

Derivations. — We derive the reduced Langevin equa- 
tion (TT~5T > using the multi-scale analysis. The Fokker-Planck 
equation corresponding to Eq. (fT2l may be written by the 
rescaled variable Y = ay, 

d,p, = -y- l d x ((-d x u - d Y v) p, - Td x p) - y - l d Y ((d x v)p t ) 

- a 2 y- l d Y (-(d Y U)P, - T8 y p) . (29) 

Since the first line of the right hand side of Eq. (f29l is the or- 
der of 0{rf) as compared with the second line, the first line can 
be treated as the perturbative terms in the limit of 77 — > 0. Ac- 
cording to the multi-scale analysis, two dimensionless times, 
t s = rjt/Ty and t y - t/r y , are introduced, and the probability 
density function is expanded as, 



P,(x, Y) = Pf],(x, Y) + 0{rf). 



(30) 



Pf\ obeys the leading order equation in Eq. 



V<>, = -a 2 y- x d Y {-{d Y U)Pf^ - Td Y Pf^ t ) , (31) 

and converges to a local stationary density, P, <()) = 
pf\x)j^ exp(-f//r) in a time scale longer than r v 

[Eq. OH], where pf\x) = J Pf\x, Y)dY. In such time scale, 
the equation obtained from the integral over y in the next or- 
der, 

rZ l {d t PZ y {x) + d t P^ ty (x)) 

= -y-% (d x T InZ(x) + //(*) - Td x ) P^(x), (32) 
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gives the Fokker-Planck equation equivalent to Eq. (T5[ in 
the limit of rj — » 0, since the left hand side of Eq. (|32] > is 
d t P t (jt)(l + 0(ij)). 
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